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09/590,075 



Examiner 

Haresh Patel 



Applicant(s) 

UEDA ET AL 



Art Unit 

2126 



- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

• Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- tf the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days wilt be considered timely. 

• tf NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C, § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)M This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) I3 Claim(s) 32-37 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) S Claim(s) 32-37 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) I3 The specification is objected to by the Examiner. 

10) [3 The drawing(s) filed on 09 June 2000 is/are: a)D accepted or b)S objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

11) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 1 1 9 and 1 20 

13) S Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)l3 All b)D Some*cO None of: 

1 .E3 Certified copies of the priority documents have been received. 

2.D Certified copies of the priority documents have been received in Application No. . 



3.D Copies of the certified copies of the priority documents have been received in this National Stage 
application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 1 9(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121 . 

Attachment(s) 

1) IS] Notice of References Cited (PTO-892) 4) O Interview Summary (PTO-413) Paper No(s). . 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) □ Notice of Informal Patent Application (PTO-152) 

3) I3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) 6 . 6) □ Other: 



U.S. Patent and Trademark Office 
PTOL-326 (Rev. 04-01) 
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DETAILED ACTION 



Claims 32-37 are presented for examination. 



Priority 



Acknowledgment is made of applicant's claim for foreign priority under 35 



U.S.C 119(a)-(d). 



Specification 



3. The following guidelines illustrate the preferred layout for the specification of a utility 
application. These guidelines are suggested for the applicant's use. 



As provided in 37 CFR 1 .77(b), the specification of a utility application should include 
the following sections in order. Each of the lettered items should appear in upper case, without 
underlining or bold type, as a section heading. If no text follows the section heading, the phrase 
"Not Applicable" should follow the section heading: 

(a) TITLE OF THE INVENTION. 

(b) CROSS-REFERENCE TO RELATED APPLICATIONS. 

(c) STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 

DEVELOPMENT. 

(d) INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A 

COMPACT DISC (See 37 CFR 1 .52(e)(5) and MPEP 608.05. Computer program 
listings (37 CFR 1.96(c)), "Sequence Listings" (37 CFR 1.821(c)), and tables 
having more than 50 pages of text are permitted to be submitted on compact 
discs.) or 

REFERENCE TO A "MICROFICHE APPENDIX" (See MPEP § 608.05(a). 
"Microfiche Appendices" were accepted by the Office until March 1, 2001.) 

(e) BACKGROUND OF THE INVENTION. 

(1) Field of the Invention. 

(2) Description of Related Art including information disclosed under 37 CFR 1.97 
and 1.98. 



Arrangement of the Specification 
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(f) BRIEF SUMMARY OF THE INVENTION. 

(g) BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S). 

(h) DETAILED DESCRIPTION OF THE INVENTION. 

(i) CLAIM OR CLAIMS (commencing on a separate sheet). 

(j) ABSTRACT OF THE DISCLOSURE (commencing on a separate sheet). 

(k) SEQUENCE LISTING (See MPEP § 2424 and 37 CFR 1.821-1.825. A "Sequence 
Listing" is required on paper if the application discloses a nucleotide or amino 
acid sequence as defined in 37 CFR 1.821(a) and if the required "Sequence 
Listing" is not submitted as an electronic document on compact disc). 

The disclosure is objected to because of the following informalities: 

i. CROSS-REFERENCE TO RELATED APPLICATIONS is missing. 
Appropriate correction is required. 

4. The title of the invention is not descriptive. A new title is required that is clearly 
indicative of the invention to which the claims are directed. 

The following title is suggested: "System and devices to improve the visibility of the 
multimedia display using additional attribute multimedia data". 



5. Applicant is reminded of the proper language and format for an abstract of the disclosure. 

The abstract should be in narrative form and generally limited to a single paragraph on a 
separate sheet within the range of 50 to 150 words. It is important that the abstract not exceed 
150 words in length since the space provided for the abstract on the computer tape used by the 
printer is limited. The form and legal phraseology often used in patent claims, such as "means" 
and "said," should be avoided. The abstract should describe the disclosure sufficiently to assist 
readers in deciding whether there is a need for consulting the full patent text for details. 

The language should be clear and concise and should not repeat information given in the 
title. It should avoid using phrases which can be implied, such as, "The disclosure concerns," 
"The disclosure defined by this invention," "The disclosure describes," etc. 

a. The abstract should not contain than 1 50 words. 
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b. 



The abstract should not contain only one paragraph. 



Appropriate correction is required. 



Drawings 



6. Figure 1 should be designated by a legend such as -Prior Art- because only that which is 
old is illustrated. See MPEP § 608.02(g). A proposed drawing correction or corrected drawings 
are required in reply to the Office action to avoid abandonment of the application. The objection 
to the drawings will not be held in abeyance. 

Information Disclosure Statement 

7. An initialed and dated copy of Applicant's IDS form 1449, Paper No. 6, is attached to the 
instant Office action. 



8. The following is a quotation of the appropriate paragraphs of 35 U.S. C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 2 1(2) of such treaty in the English language. 



Claim Rejections - 35 USC §102 



9. Claims 32-37 are rejected under 35 U.S.C. 102(e) as being anticipated by Nelson et al. 
6,243,713 (Hereinafter Nelson). 
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10. As per claims 32 and 37, Nelson teaches the following: 

a broadcast data receiving device for receiving and outputting broadcast data including a 
plurality of multimedia data (e.g., a system, method and software product provides for true 
multimedia document retrieval by receiving multimedia queries containing various types of data, 
such as text keywords, images, audio or other data types, and processing such queries against a 
multimedia index that commonly indexes multimedia documents, including any of their 
multimedia components, abstract), said data device comprising: 

a receiving unit operable to sequentially receive the plurality of multimedia data 
included in the broadcast data (e.g., a system, method and software product provides for true 
multimedia document retrieval by receiving multimedia queries containing various types of data, 
such as text keywords, images, audio or other datatypes, and processing such queries against a 
multimedia index that commonly indexes multimedia documents, including any of their 
multimedia components, abstract), 

an outputting unit operable to output the received multimedia data (e.g., a system, 
method and software product provides for true multimedia document retrieval by receiving 
multimedia queries containing various types of data, such as text keywords, images, audio or 
other data types, and processing such queries against a multimedia index that commonly indexes 
multimedia documents, including any of their multimedia components, abstract), 

a storing unit operable to create and store management information for collectively 
managing the plurality of received multimedia data (e.g., The unified index allows query 
retrieval by evaluation of a query structure which can contain any of the multimedia data types, 
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and operators which can be evaluated on any of these data types. The system indexes 
multimedia documents by separating them into their multimedia components, and processing 
each component into a number of tokens. The tokens are stored in the index along with 
information identifying the documents that contain the token, and reference data describing the 
position of the token within the documents, and any other data extracted from the multimedia 
component of the document, such as color, texture, luminance, recognized speech, or the like. 
During retrieval, a query is decomposed into multimedia components, which are then converted 
to a set of tokens and query structure including mathematical and proximity operators. Query 
expansion is used to expand the query structure to include additional tokens corresponding to 
various ones of input query tokens. Because the multimedia components are all indexed in the 
unified index, there is no need to process different parts of the query against different indices and 
databases in order to select documents that best satisfy the query, abstract) and attribute 
information corresponding to respective multimedia data (e.g., As an optional process to increase 
the robustness of the multimedia retrieval pipeline, type-specific query tokens may be added 180 
to any or all of the components that are in the query. Query expansion 180 selects "alternate" 
tokens to add to the query based on the original query tokens. For example, additional tokens 
may be used to represent other words similarly spelled to query keywords or that have similar 
meanings, or other images (or image attributes) of similar shape or color, texture, and so forth. 
This expansion can either be done by default, or at the discretion of the user via query operators. 
In addition, query expansion can add tokens of one component type in response to the presence 
of tokens of another type, col. 7, lines 1 1-23), and 
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a managing unit operable to manage the received multimedia data with reference to the 
management information (e.g., a system, method and software product provides for true 
multimedia document retrieval by receiving multimedia queries containing various types of data, 
such as text keywords, images, audio or other data types, and processing such queries against a 
multimedia index that commonly indexes multimedia documents, including any of their 
multimedia components, abstract). 

11. As per claim 33-36, Nelson teaches the following: 

the attribute information is included in the broadcast data (e.g., image attributes, col. 7, 
lines 11-23), 

the attribute information includes information indicating data type of the multimedia data 
(e.g., image attributes, col. 7, lines 1 1- 23), and wherein said managing unit is further operable to 
refer to the management information and change a process for outputting the received 
multimedia data according to the data type (e.g., a system, method and software product provides 
for true multimedia document retrieval by receiving multimedia queries containing various types 
of data, such as text keywords, images, audio or other datatypes, and processing such queries 
against a multimedia index that commonly indexes multimedia documents, including any of their 
multimedia components, abstract), 

the datatype includes at least an HTML format (HTML documents, col. 8, line 23), 
the attribute information includes start-up information of the multimedia data (e.g., As an 
optional process to increase the robustness of the multimedia retrieval pipeline, type-specific 
query tokens may be added 180 to any or all of the components that are in the query. Query 
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expansion 180 selects Alternate" tokens to add to the query based on the original query tokens. 
For example, additional tokens may be used to represent other words similarly spelled to query 
keywords or that have similar meanings, or other images (or image attributes) of similar shape or 
color, texture, and so forth. This expansion can either be done by default, or at the discretion of 
the user via query operators. In addition, query expansion can add tokens of one component type 
in response to the presence of tokens of another type, col. 7, lines 1 1-23). 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

See Form PTO-892. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Haresh Patel whose telephone number is (703) 605-5234. The 
examiner can normally be reached on Monday-Friday from 8:00 am to 5:30 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Follansbee, can be reached at (703) 305-8498. 

The appropriate fax phone number for the organization where this application or 
proceeding is assigned is (703) 306-5404. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 



Haresh Patel 



September 6, 2003. 




